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FOREWORD 


This  report  describes  the  development  and  evaluation  of  an  improved  selector 
composite  that  predicts  an  applicant's  likely  academic  and  military  performance  at  the 
Naval  Academy  r..ore  accurately  than  previous  composites.  Although  Nava)  Academy 
students  are  already  of  superior  caliber,  the  use  of  this  composite  Is  expected  to  result  in 
further  improvement  in  quality. 

This  effort  was  conducted  in  response  to  a  request  by  U.S.  Navai  Academy  to  provide 
continuous  monitoring  and  validation  of  the  midshipmen  selection  procedures. 


JOHN  3 .  PASS 
Director,  Personnel  Systems  Department 
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SUMMARY 


Problem 


An  empirically-developed  composite  called  the  Candidate  Multiple  has  been  used  by 
the  Naval  Academy  since  1975  to  aid  in  selecting  midshipmen.  Since  the  composite  was 
implemented,  several  events  have  made  it  desirable  to  investigate  re-weighting  the 
predictors  to  improve  prediction.  First,  data  is  now  available  to  correct  for  the  effects  of 
the  selection  process  itself  when  determining  the  optimal  weights;  and  second,  average 
SAT  scores  have  replaced  single  highest  SAT  scores  in  the  composite,  altering  the 
relationships  of  predictor  variables  to  each  other  and  most  likely  altering  the  optimal 
predictor  weights. 

Objective 

The  purpose  of  this  research  is  to  investigate  the  empirical  re-weighting  of  predictors 
in  the  Candidate  Multiple  with  the  intent  of  improving  the  Multiple's  ability  to  predict 
academic,  military,  and  attrition  criteria  at  the  Naval  Academy. 

Approach 

The  Candidate  Multiple  includes  seven  predictor  variables  that  are  used  to  predict 
academic  and  military  performance  as  well  as  attrition  at  the  Academy.  These  predictor 
variables  include  verbal  and  mathematics  subtests  of  the  SAT,  rank  in  high  school  class, 
high  school  teachers'  recommendations,  extracurricular  activities,  and  two  scores  derived 
from  a  vocational  interest  inventory.  To  investigate  alternative  methods  of  weighting 
these  variables  to  predict  the  criteria,  applicants  to  the  Classes  of  1983,  1984,  ana  1987 
(N  s  8,264)  were  used  to  correct  predictor-criterion  correlations  for  the  effects  of 
selection.  These  corrected  correlations  were  then  applied  to  selectees  in  the  same  classes 
(N  r  2,835)  to  derive  four  experimental  selector  composites.  The  composites  differed  in 
terms  of  which  predictor  weights  were  pre-detei  mined  by  the  Academy  for  policy  reasons 
and  which  were  empirically  optimized.  The  four  experimental  composites  were  then 
cross-validated  on  the  Academy  Class  of  1988  and  compared  with  earlier  operational 
versions  of  the  Candidate  Multiple. 

Results  and  Conclusions 


All  four  of  the  experimental  composites  were  more  valid  than  the  previous  opera¬ 
tional  composites  for  predicting  academic  performance  and  academic  disenrollment.  As 
expected,  improvement  in  predicting  these  criteria  was  greatest  when  all  variables  were 
free  to  assume  empirically-determined  weights  and  decreased  as  more  constraints  were 
placed  on  the  empirical  weighting  process. 

Recommendations 

Modify  predictor  weights  used  in  the  Candidate  Multiple  to  reflect  the  weights 
empirically  derived  in  this  research.  (The  Academy  did,  in  fact,  adopt  a  modification  of 
one  of  the  experimental  composites  for  use  with  the  Class  of  1990.)  Continue  research  to 
expand  the  range  of  criteria  that  can  be  predicted  by  the  Candidate  Multiple. 


vii 


CONTENTS 


Page 


INTRODUCTION  .  1 

Problem  . 1 

Objectives .  ) 

PROCEDURES  .  1 

I  Subjects  . . . . .  1 

\  Predictors  , . 

'  Criteria  . 

'  Statistical  Analyses . 

!  RESULTS  AND  DISCUSSION  .  4 

i 

[i 

>  Applicant  Data . . . . . . . . .  4 

■  Zero-order  Validities . 4 

ti  Corrected  Validities  . 4 

jj  Composite  Validities  7 

I;  CONCLUSIONS  AND  RECOMMENDATIONS  .  10 


i; 


i 


REFERENCES  .  11 

DISTRIBUTION  LIST  .  13 


LIST  OF  TABLES 


1.  Predictor  Intercorrelations,  Means,  and  Standard  Deviation  (SD) 
for  N  =  8,264  Applicants  to  the  Naval  Academy  Classes  of  1983, 

1984,  and  1987  for  Whom  Average  SAT  Scores  Were  Available  .  .  5 

2.  Predictor  Intercorrelations,  Means,  Standard  Deviations  (SD) 
and  Validities  for  N  =  2,835  Students  in  the  Naval  Academy 

Classes  of  1983,  1984,  and  1987  .  6 

3.  Corrected  and  Uncorrected  Validities  for  the  1983,  1984, 

and  1987  USNA  Classes  . . .  8 

4.  Cross-Validation  of  Experimental,  Operational  and  Current  Selection 

Composites  on  the  USNA  Class  of  1988  at  the  End  of  Plebe  Year .  9 


IX 


Vt<  WN 


INTRODUCTION 


Problem 


For  a  number  of  years,  selection  of  midshipmen  for  the  Naval  Academy  has  been 
based  largely  on  a  composite  score  known  as  the  Candidate  Multiple.  Initially,  the 
weights  attached  to  each  of  the  variables  in  the  composite  were  rationally  determined  by 
expert  judges  and  rationally  revised  and  updated  from  time  to  time. 

In  1975,  as  requested  ty  the  Superintendent,  U.S.  Naval  Academy,  the  empirical 
weighting  of  predictors  was  instituted  toward  the  goal  of  improving  the  efficiency  of  tne 
selection  composite  for  predicting  academic,  military,  and  disenrollment  criteria  at  the 
Academy  as  well  as  choice  of  major. 

Since  1975,  several  factors  have  made  it  desirable  to  revise  the  Candidate  Multiple. 
First,  data  is  now  available  to  improve  estimation  of  the  influence  of  pre-selection 
(restriction  of  range)  on  the  effectiveness  of  the  components  of  the  Candidate  Multiple. 
Second,  research  conducted  for  the  Naval  Academy  by  the  Navy  Personnel  Research  and 
Development  Center  (NAVPERSRANDCEN)  has  indicated  that  prediction  of  academic 
criteria  can  be  improved  in  cases  where  there  are  multiple  administrations  of  the 
Scholastic  Aptitude  Test  (SAT)  or  American  College  Test  (ACT)  using  the  average  of  each 
candidate’s  scores,  rather  than  the  highest  such  score  (Cowen  &  Abrah.  ms,  1982).  Other 
things  being  equal,  use  of  the  average  score  (implemented  with  the  class  of  1987)  should 
influence  the  weights  that  SAT  and  other  variables  receive  in  the  Candidate  Multiple. 

Objectives 

The  purpose  of  this  research  was  to  investigate  a  revision  of  the  weights  assigned  to 
predictors  in  the  Candidate  Multiple  to  further  improve  the  efficiency  of  the  selection 
composite.  Specific  objectives  were  (1)  to  optimally  weight  the  individual  predictors 
against  academic,  military  and  attrition  criteria  at  the  Academy,  and  (2)  to  counteract 
the  influence  of  pre-selection  when  deriving  the  optimal  weights. 


PROCEDURES 


Subjects 

Subjects  included  both  applicants  and  selectees  from  the  Naval  Academy  classes  of 
1983,  1984,  and  1987,  and  selectees  only  from  the  class  of  1988.  The  applicants,  totalling 
8,264  individuals,  were  included  to  permit  corrections  for  the  effects  of  selection  on 
validity  coefficients.  The  selectees  were  required  for  development  (classes  of  1983,  1984, 
and  1987)  and  cross-validation  (class  of  1988)  of  the  revised  predictor  weights  in  the 
Candidate  Multiple.  Selectees  were  distributed  among  the  four  class  grouos  as  follows: 
1983  (N  =  1,061),  1984  (N  =  945),  1987  (N  =  829),  and  1988  (N  =  1,192). 

The  four  year  groups  were  chosen  because  they  were  among  the  most  recent  to  have 
(1)  complete  records  of  all  SAT  and  ACT  that  were  needed  to  compute  average  SAT  score 
values,  and  (2)  the  most  recent  versions  of  interest  test  scores.  (The  class  of  1985  was 
omitted  because  it  was  missing  complete  SAT  information  and  the  class  of  1986  because  it 
was  missing  the  interest  test  scores  used  in  the  present  multiple.)  Several  year  groups 
were  included  to  ensure  that  results  obtained  would  be  stable  over  time. 


Predictors 


The  following  predictor  scores  were  available  for  all  four  classes  and  for  the 
applicant  sampiest 

1.  Scholastic  Aptitude  Test— Verbal  (SATV)  or  its  empirically-derived  ACT 
equivalent.  This  score  represents  the  verbal  aptitude  of  the  individual  as  measured  in  a 
national  competitive  testing  program  designed  for  college  admissions  and  scholarship 
awards.  For  the  classes  in  this  sample,  a  minimum  score  of  520  was  required  for  Naval 
Academy  qualification.  In  exceptional  cases,  waivers  have  been  allowed  for  those  scoring 
below  520.  For  each  Individual  who  took  the  SAT  and/or  ACT  more  than  once,  an  average 
SATV  score  was  computed. 

2.  Scholastic  Aptitude  Test— Mathernatics  (SATM)  or  its  empirically-derived  ACT 
equivalent!  This  score  is  parallel  to  SATV  except  that  it  represents  an  individual's 
quantitative  aptitude.  Since  the  curriculum  at  the  Naval  Academy  is  heavily  weighted 
with  engineering  and  science  courses,  the  minimum  qualifying  SATM  score  for  the  four 
classes  in  this  study  was  600.  Waivers  were  allowed  in  exceptional  instances  for  those 
scoring  below  this  value.  As  with  SATV,  an  average  SATM  score  was  computed  for  each 
individual  who  took  the  SAT  and/or  ACT  more  than  once. 

3.  Rank  in  Class  (R/C).  This  is  a  standardized  score  (x  *  500,  S.D.  =  100),  which  is 
based  on  an  individual's  high  school  rank  and  ranges  between  200  and  800. 

4.  Recommendations  (REC).  This  is  a  score  based  on  high  school  officials' 
estimates  of  the  individual's  potential  for  success  as  a  naval  officer.  These  officials 
(normally  English  or  Mathematics  teachers)  are  asked  to  evaluate  the  candidate  on 
effective  communication  skills,  interpersonal  relations,  personal  conduct,  and  leadership 
potential.  A  score  derived  by  summing  these  evaluations  may  range  from  0  to  1,000,  with 
no  minimum  qualifying  score  required. 

5.  Extracurricular  Activities  (ECA).  Each  applicant  is  asked  to  complete  the 
Candidate  Activities  Record,  a  rather  extensive  form  that  covers  participation  in  both 
athletic  and  non-athletic  activities  during  high  school  An  objective  scoring  system  (not 
empirically  derived)  is  used  to  compute  an  Extracurricular  Activities  (ECA)  score  that 
ranges  from  300  to  800.  No  minimum  qualifying  score  is  required. 

6.  Strong  Campbell  Interest  Inventory  (SCII).  This  is  a  vocational  interest  question¬ 
naire  from  which  scales  have  been  constructed  to  predict  criteria  such  as  voluntary 
resignation,  military  aptitude  rating,  and  choice  of  major  (Abrahams  &  Neumann,  1973; 
Neumann  &  Abrahams,  1974).  Two  of  these  scales  were  included  as  predictors: 

a.  Career  Retention  Scale  (CRS).  The  CRS  comprises  SCII  item  responses 
keyed  to  differentiate  between  high-tenure  and  low-tenure  officers. 

b.  Engineering-Science/Humanities-Social  Science  Scale  (ES/HSS).  The  SCII 
items  are  keyed  on  this  scale  to  identify  applicants  with  high  interest  in  Engineering- 
Science,  or  high  interest  in  Humanities-Social  Sciences.  Thus,  individuals  with  high 
interest  in  either  one  of  these  Academy  major  areas  can  be  identified. 
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Criteria 

Five  primary  criteria  were  included  as  measures  of  Academy  successi 

1.  Academic  Quality  Point  Ratio  (AOPR).  The  AQPR  used  In  this  study  is  the 
cumulative  grade-point  average  earned  by  a  midshipman  after  2  years  at  the  Academy.  If 
the  midshipman  disenrolled  any  time  within  the  first  2  years,  his  or  her  lest  available 
AOPR  was  used. 

2.  Military  Quality  Point  Ratlna  (MQPR).  The  MQPR  is  a  composite  rating  assigned 
to  midshipmen  at  tne  end  of  every  semester.  It  is  a  weighted  composite  based  on  grades 
received  in  physical  education,  professional  competency  review,  military  performance, 
conduct,  and  in  professional  courses.  This  rating  is  of  special  importance  since  research 
conducted  at  the  Academy  (Howland,  1970)  and  more  recently  at  NAVPERSRANDCEN  has 
shown  that  a  similar  measure  is  related  to  later  officer  performance. 

For  the  purpose  of  this  research,  a  cumulative  MQPR  was  computed  for  each 
midshipman,  based  on  his  or  her  first  2  years1  ratings  at  the  Academy.  As  with  the  AQPR, 
if  an  individual  disenrolled  any  time  within  the  first  2  years,  the  last  available  MQPR  was 
used. 

3.  Voluntary  Resignation  (VOL  RES).  During  the  first  2  years,  a  midshipman  may 
voluntarily  withdraw  from  the  Academy.  There  may  be  a  number  of  reasons  for  this 
decision,  such  as  personal  or  family  problems,  disenchantment  with  a  naval  career,  or 
poorer  performance  than  the  midshipman  is  willing  to  tolerate.  Whatever  the  reason,  this 
criterion  has  been  the  least  predictable  of  those  under  consideration. 

4.  Academic  Disenrollment  (ACAD).  A  midshipman  may  be  discharged  for  failing 
to  meet  academic  standards  at  the  Academy.  Currently,  about  seven  percent  of  a  class 
are  separated  for  academic  reasons. 

5.  All  Disenrollment  (ALL  PIS).  Since  any  decrease  in  the  overall  disenrollment 
rate  would  be  desirable,  the  prediction  of  this  criterion  was  examined  separately.  This 
measure  includes  all  midshipmen  who  separate  from  the  Academy  either  voluntarily  or 
involuntarily,  with  the  exception  of  medical  discharges.  The  latter  are  not  included  since 
it  seems  highly  unlikely  that  such  separations  would  be  predictable  from  the  variables 
being  investigated  here.  In  the  most  recent  graduating  class  (1987),  about  19  percent 
disenrolled  voluntarily  or  involuntarily  within  the  first  2  years,  and  approximately  23 
percent  disenrolled  by  the  end  of  the  4-year  program. 

Statistical  Analyses 

Means,  standard  deviations,  and  intercorrelations  among  the  seven  predictor  variables 
were  obtained  for  the  8,264  applicants  to  the  Naval  Academy  classes  of  1983,  1984,  and 
1987  for  whom  complete  data  were  available  on  all  predictors  used  in  the  current 
selection  composite.  These  statistics,  together  with  zero-order  validities  of  seven 
predictors  against  the  five  criteria,  were  also  computed  for  students  in  the  Naval 
Academy  classes  of  1983,  1984,  and  1987  (N  =  2,835)  for  whom  complete  data  were 
available.  The  resulting  applicant  and  class  data  was  then  used  to  correct  the  zero-order 
validities  tor  restriction  in  range.  (See  Guilford,  1965,  p.  344,  for  the  correction  formula.) 

Using  the  corrected  validity  values,  multiple  regression  analyses  were  performed  to 
obtain  the  optimal  weighting  of  the  seven  predictors  for  each  of  the  five  criteria.  All 
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predictors  were  evaluated  against  all  criteria  despite  the  fact  that  some  of  he  predictors 
would  be  much  more  likely  to  predict  non-academic  criteria  such  as  choice  of  major  or 
career  retention- -criteria  that  will  be  considered  in  future  research.  Once  determined) 
the  optimal  weights  obtained  against  each  criterion  were  combined  statistically  to  predict 
an  equally-weighted  combination  of  academic  performance,  military  performance,  and 
disenroilment.  The  equations  resulting  from  these  weighting  procedures  were  evaluated 
against  each  of  the  criteria  using  the  class  of  1988,  and  were  compared  to  previous 
operational  composites  and  to  equations  in  which  weights  were  adjusted  to  reflect 
judgmental  criteria  provided  by  the  Academy. 


RESULTS  AND  DISCUSSION 


Applicant  Data 

Table  1  presents  the  individual  predictor  and  Candidate  Multiple  intercorrelations, 
means,  and  standard  deviations  for  the  8,260  applicants  for  whom  all  predictor  scores, 
including  average  SAT,  were  available. 

Zero-order  Validities 


Zero-order  validities  were  computed  for  all  predictors  against  each  criterion  at  the 
end  of  2  years  of  school  for  2,835  students  in  the  validation  sample.  Table  2  presents 
these  validities  as  well  as  the  predictor  intercorrelations,  means,  and  standard  deviations. 
It  is  of  interest  to  note  that  the  average  SATM  is  the  best  predictor  of  AQPR  and  ACAD 
D1S,  while  MQPR  is  predicted  equally  well  by  SATM  and  R/C.  None  of  the  individual 
predictors  have  very  high  validity  against  VOL  RES  or  against  ALL  DIS.  The  Candidate 
Multiple,  however,  is  effective  against  all  criteria  except  VOL  RES. 

As  expected,  the  last  four  non-academic  predictors  (RECS,  ECA,  ES/HSS,  CRS)  are 
not  effective  in  predicting  the  essentially  academic  criteria.  Work  is  continuing  with 
these  measures  to  improve  the  prediction  of  other  criteria  not  presently  addressed  by  the 
Candidate  Multiple,  such  as  non-academic  attrition  and  post-Academy  leadership 
performance. 

Corrected  Validities 


When  tests  are  used  in  selection,  their  statistics  are  often  distorted  by  the  selection 
process.  For  example,  the  average  test  score  for  selectees  is  higher  than  the  average  test 
score  for  applicants,  because  those  with  low  scores  are  usually  eliminated  in  the  selection 
process.  Less  obviously,  the  validities  of  tests  used  in  selection  are  lower  in  selectees 
than  in  applicants  because  of  less  variability  in  the  scores  of  the  selectee  group.  By 
examining  corrected  validities  for  such  restriction,  in  range,  a  more  accurate  evaluation 
of  how  effective  the  different  predictors  are  at  estimating  the  relevant  criteria  is 
obtained.  A  more  important  reason  for  examining  corrected  validities  is  to  see  whether 
the  relative  magnitudes  of  the  validities  shift  after  correction.  Often,  the  validities  of 
operational  predictors  that  receive  the  most  weight  are  more  affected  by  selection  than 
are  the  validities  of  operational  predictors  receiving  less  weight.  Experimental  predictors 
(i.e.,  those  not  used  to  screen  applicants)  usually  are  least  affected  by  the  selection 
process. 
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Predictor  Intercorrelations,  Means,  and  Standard  Deviations  (SD) 
for  N  =  8,264  Applicants  to  the  Naval  Academy  Classes  of 
1983,  1984,  and  1987  for  Whom  Average  SAT 
Scores  Were  Available 
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Table  3  presents  the  corrected  and  uncorrected  validities  of  the  individual  predictors 
against  all  five  criteria  for  the  validation  samples.  As  expected,  the  corrected  validities 
are  generally  higher  than  the  uncorrected  validities,  especially  for  the  academic  pre¬ 
dictors  and  criteria.  However,  there  was  no  major  shift  in  the  order  of  the  corrected 
validities. 

Composite  Validities 

The  corrected  validities  and  intercorrelations  based  on  the  1983,  1984,  and  1987 
classes  presented  in  in  Tables  2  and  3  were  used  to  develop  several  experimental 
candidate  multiple  composites,  one  of  which  was  modified  slightly  by  the  Academy  and 
implemented  operationally  as  the  Candidate  Multiple  for  the  class  of  1990  (CM90).  The 
experimental  Candidate  Multiples  (CM90A,  CM90B,  CM90C,  CM90D)  and  the  operational 
Candidate  Multiples  for  the  classes  of  1990,  1989,  and  1988  (CMO,  CM89,  CM88),  were 
cross-validated  on  selectees  from  the  Academy  class  of  1988.  These  validity  data  are 
presented  in  Table  4. 

CM90A.  In  the  first  experimental  Candidate  Multiple,  CM90A,  the  predictor 
variables  were  weighted  optimally  to  predict  the  equally-weighted  sum  of  three  criteria: 
AQPR,  MQPR,  and  ALL  DIS  (all  disenrollment).  ALL  DIS  was  included  so  that  any 
predictable  variance  in  academic,  honor  or  conduct  discharges  or  in  voluntary  resignations 
would  be  captured  in  the  least  squares  regression  process.  CM90A  represents  the 
maximum  attainable  validity  against  the  composite  criterion  for  a  weighted  sum  of  the 
selection  instruments  as  they  are  presently  scored.  Although  this  optimal  weighting  failed 
to  predict  two  criteria  of  interest  to  the  Academy  (i.e.,  VOL  RES  and  ALL  DIS),  it  did 
provide  a  standard  against  which  to  measure  other  experimental  candidate  multiple 
weightings. 

CM90B.  The  second  experimental  Candidate  Multiple,  CM90B,  examined  the  optimal 
weighting  of  the  last  five  predictors  under  the  constraint  that  SATV  and  SATM  would 
obtain  equal  weight.  The  equal  weighting  of  SATV  and  SATM  reflected  the  Academy's 
desire  to  obtain  a  composite  that  did  not  unduly  favor  engineering-  and  science-related 
abilities  and  propensities  over  their  Humanities-related  counterparts.  It  can  be  seen  that 
this  weighting  results  in  slightly  lower  validities  for  AQPR  and  ACAD  DIS.  These 
reductions,  although  statistically  significant,  are  of  little  practical  consequence. 

CM90C.  The  third  experimental  composite,  CM90C,  maintained  a  set  of  weights 
specified  by  the  Academy  (and  shown  in  Table  4)  for  the  last  four  predictors  and  allowed 
the  weights  for  SATV,  SATM,  and  R/C  to  vary  freely.  This  experimental  composite  had 
validities  almost  equal  to  those  attained  by  the  optimal  composite  (CM90A),  but  SATM 
received  more  than  twice  the  weight  of  SATV.  While  CM90C  had  acceptable  validities, 
the  resulting  optimal  weights  for  SATV  and  SATM  did  not  meet  the  requirement  of 
treating  humanities-  and  engineering/sciences-related  abilities  and  propensities  equally. 

CM90D.  The  fourth  experimental  composite,  CM90D,  maintained  both  the  Academy- 
specified  weights  for  the  last  four  predictors  and  equal  weights  for  SATV  and  SATM.  With 
these  restrictions,  the  relative  weights  of  SATV,  SATM,  and  R/C  were  chosen  to  optimize 
the  validities.  This  fourth  experimental  composite  was  slightly  less  valid  than  the  other 
three  composites  for  predicting  AQPR  and  ACAD  DIS,  but  maintained  desirable  relative 
weights  for  all  variables. 
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CM90.  The  operational  composite,  CM90,  was  developed  in  collaboration  with 
Academy  officials  so  as  to  consider  criteria  other  than  those  for  which  objective 
information  was  readily  available.  This  operational  weighting  gives  slightly  greater 
weight  to  SAT  than  does  CM90D  while  weighting  SATV  and  5ATM  equally.  At  the  same 
time,  it  maintains  the  relative  weights  desired  by  the  Academy  for  the  last  four 
predictors.  This  operational  composite  is  just  as  valid  as  CM90D  in  the  cross-validation 
sample,  while  giving  the  desired  emphasis  to  SAT. 

The  weights  for  CMSS  and  CM89,  presented  in  Table  4,  show  the  increasing  emphasis 
over  time  on  the  verbal  component  of  SAT.  It  can  also  be  seer;  that  the  current 
operational  composite,  CM90,  continues  this  progressive  emphasis  without  sacrificing 
validity.  In  fact,  CM90  has  significantly  higher  validity  than  either  CM88  or  CM89  for 
predicting  most  criteria  in  the  1988  sample.  These  results  suggest  that  validity  will  not 
be  sacrificed,  and  may  even  be  improved,  in  reaching  the  Academy's  goals. 


CONCLUSIONS  AND  RECOMMENDATIONS 

All  four  of  the  experimental  composites  improved  on  the  prediction  of  AQPR,  MQPR, 
and  academic  disenrollment  when  compared  to  previous  operational  composites.  Improve¬ 
ment  was  slightly  greater  for  the  academic  criteria  when  all  variables  were  free  to 
assume  empirically-determined  weights,  and  decreased  as  more  constraints  were  placed 
on  the  empirical  weighting  process. 

Based  on  these  results,  it  is  recommended  that  the  predictor  weights  used  in  the 
Candidate  Multiple  be  modified  to  reflect  the  weights  empirically  derived  in  this 
research.  (The  Academy  did,  in  fact,  adopt  a  modification  of  one  of  the  experimental 
composites  for  use  with  the  class  of  1990.)  It  is  also  recommended  that  research  continue 
to  expand  the  range  of  criteria  that  can  be  predicted  by  the  Candidate  Multiple. 
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